CVLNet: Cross-view Semantic Correspondence Learning for Video-Based Camera Localization

This paper tackles the problem of Cross-view Video-based camera Localization (CVL). The task is to localize a query by leveraging information from its past observations, i.e., continuous sequence images observed at previous time stamps, and matching them large overhead-view satellite image. critical challenge this learn powerful global feature descriptor for sequential ground-view while considering domain alignment with reference images. For purpose, we introduce CVLNet, which first projects into an overhead view exploring ground-and-overhead geometric correspondences then leverages photo consistency among projected form representation. In way, cross-view differences are bridged. Since usually pre-cropped regularly sampled, there always misalignment between location image center. Motivated this, propose estimating camera’s relative displacement before similarity matching. estimation process, also consider uncertainty location. example, unlikely be on top trees. To evaluate performance proposed method, collect Google Map KITTI dataset construct new video-based localization benchmark dataset, KITTI-CVL. Extensive experiments have demonstrated effectiveness over single image-based superiority each module other alternatives.